Automated Integration Of Potentially Hazardous Open Systems

I speculate on the feasibility of open systems that self-assemble into integrated systems of systems using automation to identify and manage novel hazards. 1. Automated Integration of Open Systems One of the benefits expected of open systems is that they can be combined as systems of systems to deliver some integrated service beyond that provided by any of the constituents alone. Of course, we would like some assurance that the integrated service accomplishes what we require and, perhaps more importantly, that it maintains safety, or other critical properties. Furthermore, we can imagine that these integrated systems of systems are constructed dynamically (i.e., during operation), either under the control of some “integrating app,” or spontaneously as the constituent system discover each other (e.g., when multiple medical devices are attached to a single patient). We then need the assurance of required and safe behavior to be constructed automatically and dynamically also. One way to accomplish dynamic integration and assurance is for the individual systems to be supplied with models of their properties, assumptions and behavior, and an argument providing assurance that they deliver their required and critical properties. As systems integrate into systems of systems, they exchange their models and assurance case arguments and compose these into a larger model and argument for the integrated system. The first step (exchange and composition of models) is an emerging framework known as Models@Runtime (M@RT) [1] while the second step (exchange and synthesis of assurance arguments) is known as Safety Models@Runtime (SM@RT) [2]. Trapp and Schneider [2] distinguish four levels of sophistication and difficulty in SM@RT according to how ambitious is the integration, and note that only the first two are feasible at present. My interpretation of this four-level hierarchy, starting with the simplest case, is the following. The focus here is on safety, but the ideas can be generalized to other critical properties, or to conventional requirements. Unconditionally safe integration. Here, the component systems guarantee their own safety, with no assumptions on their environment. It follows that when two or more such systems are integrated into a system of systems, the result is also unconditionally safe. Trapp and Schneider refer to this class of systems as “Safety Certificates at Runtime.” Conditionally safe integration. Here, the component systems guarantee their own safety, but do have assumptions on their environment. When two such systems are integrated into a system of systems, each becomes part of the environment of the other and it is necessary for them to exchange their models and assurance arguments and to prove that the assumptions of each are satisfied by the properties of the other. The resulting system will also be conditionally safe. Trapp and Schneider refer to this class of systems as “Safety Cases at Runtime.” Safely managed integration. This class is similar to the previous one except the component systems are not able to ensure each others assumptions. Hence one or both systems must be adapted in some way, generally by synthesizing a wrapper or runtime monitor that excludes the troublesome cases. For example, if one system delivers an unacceptable result, a runtime monitor can block it and signal failure to the other system. Or if one system cannot deliver the assumed behavior in some cases, a wrapper can block or transform its inputs to exclude those cases. Trapp and Schneider refer to this class of systems as “V&V at Runtime.” Safe integration despite hazards. In this class, it is possible that the integrated system has hazards (i.e., potentially unsafe circumstances) not present with either system individually. For example, a surgical laser may be safe and an anesthesia machine may be safe, but the combination possesses a new hazard that the laser can cause burning and fire in the enriched oxygen supplied by the anesthesia machine [3]. Once the hazards are known, this class can be transformed into the previous one (e.g., the laser can be disabled if the anesthesia machine is delivering enriched oxygen, or the anesthesia machine can be instructed not to use enriched oxygen if the laser is operating). Trapp and Schneider refer to this class of systems as “Hazard Analysis and Risk Assessment at Runtime.” The first class of integrated systems is straightforward; the second is seen in the Japanese DEOS project [4] and in the already cited work of Trapp and Schneider. The third class is anticipated in the NATO interoperation framework called SILF [5] and prototyped in an SRI project called ONISTT [6]. I outline these projects and discuss related ideas and prospects in a previous paper [7].